DOCUMENT RTSUME 



FH OOP 6P9 

Festa, Paul F. ; Others 

StrateaiGs for Develcpnient of Corn pu ter-Paset 
instructional W ana cement Systeins. 

Southwest Regional Fducational Lab., Tnalevood, 

Calif. 

Feb 71 

1?p. ; Papf^r presented at Annual ^eetinq of the 
American Educational Research Association (New Yort, 
N.Y., February 5, 1^71) 

FOPS Price t1^-S0.65 KC-?'^.2o 

Classrooms, "^Computer Oriented Programs, Curriculum, 
Decision flaking, Fducational Technology, Evaluation, 
Individualized Instruction, *Iti struction al Systems, 
n struct ional TechnolocVf Systeiris Development, Task 
Analysis, Tes^ Scoring Nachiner 

TfIMS, >^lTi struct ional f^onitoring and Manaqenert System 

Pecertlv, several systems bearing the label 
"instruct ional manaqemert systems" have beer developed and reported. 

An analysis of such sys+ems, however, reveals that most are limited 
to test-scoring and reporting functions and might best be considered 
instructional monitoring systems. State-of-the-art computer and 
educational technology is sufficiently advanced to Permit development 
and implementation of such systems. However, true instructional 
management systems cannot become a rf^alitv until strategies of 
development and evaluation of instructional decision rules are more 
advanced. Prior to the development of either tyre of system, thp 
specific components requiring computer support must be identified, 
the potential cost benefits of such support must be analyzed, end 
appropriate hardware/software configurations must be established. 
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STRATEGIES FOR DEVELOPMENT OF COMPUTER-BASED 
INSTRUCTIONAL MANAGEMENT SYSTEMS 

Paul E. Resta, Joel E, Strandberg and Edward Hirsch 
Purpose of Paper 

In this paper we will: (1) analyze the common and unique components 

of both instructional monitoring systems and instructional management 
systems; (2) identify and discuss the components of both systems for which 
computer support may be appropriate; and (3) identify the tasks required 
for development of more powerful monitoring and management systems as well 
as harnessing the computer to specific components of these systems. 

Definition of Instructional Management 

The term "instructional management" although relatively new, is 
coming into widespread use within the educational community. There is yet, 
however, no clear definition of what the term means other than the Impli- 
cation that it has something to do with the instructional decisions made 
subsequent to an analysis of quantitative or qualitative pupil performance 
Information . 

What is instructional management and how does it differ, if at all, 
from Instruction? Instruction may be defined as the process of presenting 
stimulus sequences (and eliciting responses) designed to bring specified 
learner behaviors under the control of specific stimuli. Instructional 
management, on the other hand, Is the process of making decisions such as: 

How should it be administered (for whom or what)? 

\ 

When should it be given? 

What, if any, contingencies are to be established? 
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The who in the above question consists of either a pupil or group of 
pupils to whom the teacher must provide independent instruction. The what 
consists of the instructional alternatives available to attain a specified 
objective. The how relates to the type of instruction to be provided and 
the vehicle of instruction (e.g.j teacher, aide, tutor, computer, self- 
instructional workbook, AV device, etc.). The when , how much , and for 
how long decisions are a function of a complex of variables such as the 
amount and type of practice needed by pupils at specific performance 
levels, the amount of time available for instruction in the specified 
curricular area, the availability of instructional support personnel, and 
the physical constraints of the classroom. The application of contingencies 
is also tied to many variables, intrinsic and extrinsic to the school. 

An instructional management system should have, as its nucleus, 
specific empirically-verified rules for generating answers to the above 
questions. The output of the system should be in the form of prescriptions 
for specific educational action s to be taken by the teacher. 

Instructional Management vs. Monitoring Systems 

Several systems bearing the label "instructional management system," 
have recently been developed and reported. An analysis of such systems, 
however, reveals that most are limited only to test scoring and reporting 
functions. They yield an array of information for the teacher which she 
may use in whatever manner she sees fit. Research and development of such 
systems focus primarily on the identification of the most appropriate 
formats and types of information used by the teacher, methods of improving 
the hardware/software components and the operational-logistical procedures. 
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Such systems might best be considered instructional monitoring systems 
since their function is to provide a reporting capability. In contrast, 
an instructional management system would contain the elements necessary 
to complete the feedback loop and to effect change in the instruction 
(see Figure 1). Given the response data from the unit test, an instruc- 
tional prescription would be generated indicating the use of specific 
materials. These instructional prescriptions would be generated from 
empirically developed and tested prescription rules.* The actions taken 
by the teacher relative to the use of the prescribed materials would be 
evaluated by administration of a second unit test. Performance on 
specific test items related to the prescribed material would then be used 
to assess its use by the teacher 

It is clear that state-of-the-art computer and educational technology 
is sufficiently advanced to permit development and Implementation of a 
wide variety of computer-based instructional monitoring systems. Before 
instructional management systems can become a reality, however, extensive 
research and development is required on the strategies for development and 
evaluation of instructional decision rules. 

Instructional Monitoring and Management System Component Tasks 

This section will analyze the components of a generalized Instructional 
monitoring and management system (IMMS), Identify the tasks to be performed, 
discuss and evaluate the potential contribution of computer support in the 
performance of those tasks. 

* 

Paper in AERA session C-60 
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Figure 1 is a flowchart which schematically represents a generalized 
IMMS. The entire system could operate in a manual mode but it is antici- 
pated that certain components, namely those involving data manipulation 
and rule-based decision making, will benefit from computer control. The 
other components consist of student-teacher interactions best left under 
human control. 

The computer-compatible tasks for monitoring ?nstruction are: test 

scoring, data organization and storage, and report generation. To manage 
instruction, another task must be performed: the processing of performance 

data by means of rule-based lugico-mathematical operations resulting in 
the generation of instructional prescriptions. The first task, test scoring, 
serves as the basis for the performance of all successive tasks. The 
resultant performance data must be organized and stored for convenient 
retrieval. In order to prepare notes, forms, and reports for self reference 
and external communication (to administrators, parents, and students), the 
performance data must be analyzed and further manipulated so that item, 
objective, and subject matter information is accessible on an individual, 
group, or class basis. If instructional decision rules have been estab- 
lished, they must also be applied to the performance data. This process 
will result in some type of instructional prescription. 

Computers process information. They collect, store, retrieve, compute, 
compare, and format data, in addition to performing logical operations on 
them. While they do only that which can be done manually, they are able to 
perform these tasks more accurately, reliably, and far more i^uickly than do 
people. Once programmed, they can perform the same type of task endlessly 
unless Instructed to do otherwise. 

er|c 
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With this in mind, let us examine the relative benefits of the manual 
and computer-assisted modes of operation. Following the individual dis- 
cussion of the tasks, a summary of their requirements in an automated and 
manual mode is presented in Table 1. 

Test Scoring 

Based on the present state-of-the-art, the manual mode must be used 
primarily with constructed responses unless there is a human interface 
between the tests and data input to the computer. Multiple choice tests 
can be scored by the computer. However, some preparatory work must be 
performed (i.e., inputting keys, scoring programs, etc.). This means that 
computer scoring will be efficient only when many tests have to be scored. 

It becomes particularly efficient when the same test is to be scored many 
times each year and in successive year=>. 

Data Organization and Storage 

Manual methods of storing and organizing information are cumbersome 
and may require filing stacks of paper and constantly revising record books. 
The computer has tremendous capabilities for storing, accessing, and up- 
dating data. To the extent that each bit of Information must be randomly 
accessed, and accessed as a member of several sets simultaneously, so 
does computer assistance become more necessary i;nd cost-effective. 

Descriptive Report Generation 

Computer support can save much time and effort on the part of teachers 
who desire and use complete records of their students* performance. There 
will be other teachers, however, who either do not perceive the value of 
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Manual Mode 



Automatic Mode 
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Multiple choice tests - Devise a 
scoring key and/or template for hand 
processing of each test. Con- 
structed response tests - Determine 
criteria, limits, and perhaps sample 
acceptable and unacceptable re- 
sponses for hand processing of each 
test. 



To store all response information, 
tests must either be filed or a 
sLudentXi temX response matrix must 
be constructed. If data is to be 
organized by item and objective, on 
an Individual basis or on the basis 
of group or class membership, a 
variet> of charts and tables must 
be created. 



If data has been organized on hard 
copy as outlined above, additional 
teacher reports are unnecessary. 
However, if notes, forms, and 
communique p to others are required, 
then they must be specially pre- 
pared by the teacher using raw data 
as the basis for those reports. 



In order to prescribe subsequent 
learning experiences for students 
or groups, teachers have to examine 
performance data, compare them to 
preestablished and empirically 
validated minimal performance 
standards, and make appropriate 
prescriptive decisions. Once the 
prescriptive decision has been 
made, the teacher implements the 
decision by taking some type of 
action (regrouping, changing 
instructional pace, assigning 
specific remedial treatments, or 
creating materials especially 
designed for individual students' 
learning deficiencies). 



Develop a computer program to score 
multiple choice tests. Devise scoring key. 
Input key and response data via teletype, 
punched cards, CRT, or scanner-generated 
mag tape. Run program for output of 
individuals’ scores and sub-scores. Con- 
structed response tests cannot be scored 
unless there is human interface between 
tests and data input to the computer. 



Test items must be keyed to objectives and 
student ID's must be associated v?ith 
groups, classes, etc. Additional state- 
ments can be added to the scoring program 
which will store performance data In 
matrices which will permit a flexible 
retrieval capability. In addition to 
pupil performance data, information per- 
taining to instructional alternatives and 
previously encountered instructional 
treatments must be stored. 



The program to score tests and store 
pcrfonnance data must be expanded to 
include statements which will output 
performance data in a variety of easy-to- 
read formats. Rach time a test is given 
(or on command, for that matter) single 
tost or cumulative data can be generated 
automatically in forms appropriate to the 
’leeds of teachers, pupils, parents, and 
administrators. 



Additional programming will be required to 
enable the computer to compare pupil 
performance data (single test or cumlativo 
test data) with validated Instructional 
treatment data. A further extension of 
the basic monitoring program must be 
created which would allow decision rules 
to be input to the computer. These 
statements of logical and computational 
operations, in conjunction with formating 
statements, would process performance data 
in such a way as to output prescriptive 
suggestions. If such a program accesses 
a lesson generating subroutine, that 
section of the program can generate 
individual ly- tai lored lessons automati- 
cally. 




Table 1 
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maintaining complete records of pupil performance, or who are hostile to 
the idea. The installation of computer-based educational systems will 
require new techniques of facilitating their acceptance. It is possible 
that by focusing the teacher’s attention on the students' post-instructional 
behavior (via the computer -generated reports), they might then begin 
making instructional decisions based on the data furnished. It is likely, 
however, that more systematic installation and training strategies will 
be needed to assure wide usage by teachers. 

Computer-generated reports, with varying content and format, may 
also expedite communication between teachers and parents, as well as 
between teachers and their supervisors (e.g., Individual student report 
to parent and class nummary report to principal). While such reports 
may be constructed manually, the process consumes considerable time. 

Finally, longitudinal progress reports can be more easily assembled by 
a computer than by a teacher. Thus, subtle differences and trends can be 
identified, which might not otherwise be spotted by the teacher or 
administrator. 

Rule-Based Processing and Prescription Generation 

As the assignment of effective prescriptions becomes more systematic, 
it is likely that the rules for such assignment will operate on increas- 
ingly more specific information, accumulated over time and related to a 
number of variables. At this point, computer assistance for prescriptive 
role processing will be required. 

The generation of prescriptions has, as its input, performance data 
and decision rules. The output Is a prescription - either a direct 
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action, a printed directive, or the latter, plus specially prepared 
instructional materials. If a computer Is being used for monitoring 
and rule processing, the output function should logically follow. It 
is in the actual construction or compilation of individually tailored 
lesson material, however, that the advantages of computer over manual 
methods should become evident. The computer could store performance 
data, a record of previous instructional treatments, prescriptive 
decision rules, the content domain, and formating programs. It could 
then process all the information on an individual cr group basis 
resulting in a set of instructional materials unique to the particular 
needs of each individual or group. The development of such materials 
in a manual mode could be difficult at best. 

Classroom Organization and Curriculum Considerations 

By now it should be obvious that computer support for instructional 
monitoring and management must be considered when the data processing 
tasks of the teacher become cumbersome. The unwleldlness of data 
processing is directly related to: 

1. the number of academic svibjects being managed and 
the number of objectives per subject 

2. the number of groups and/or individuals receiving 
differential instruction 

3. the number of instructional alternatives available 
for the achievement of each objective 
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4. the frequency of testing, number of objectives and 
items per test, number of students tested, and the 
specificity of test analysis 

5. the amount of cumulative data that will be used in 
performance analysis 

6. th- complexity of decision rules for the assignment 
of remediation and group reconstitution 

7. the frequency of group reconstitutions 

There is also an increased need for computer assistance in the 
monitoring and management of instruction to the extent that: (1) each 

student selects hie own goals for a variety of objectives; (2) he 
receives independent instruction and remediation when appropriate; 

(3) he interacts with a multiplicity of instructional resources at his 
own pace; and (4) he is tested frequently to determine the effects and 
adequacy of previous instt\iction and to insure that future learning 
opportunities lead to success. Some of those opportunities are 
especially designed to suit that student's interests and previously 
demonstrated abilities. 

In contrast, the development of a computer-assisted instructional 
monitoring and managemer*- system would hardly be worth the effort for 
situations where 

1, a teacher atminister.t instruction to an entire class 
or to several permanent subgroups in order to attain 
a small number of major objectives 
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2, she has at her disposal few instructional resources 
(usually one book per student per subject) 

3, she rarely tests pupil performance 

4, she makes few (other than ranking) decisions on che 
basis of the test information 

System Development Considerations 

Computer support for Instructional Management Systems can be seen 
as a tripartite developmental activity including the development of: 

(1) a data transmission network to input pupil response data (and ocher 
relevant information) and output reports to teachers, administrators, 
etc. (monitoring), and instructional prescriptions (management); 

(2) computer programs which perform the required analytic and formating 
tasks in the transformation of ’’input” to ’’output;” and (3) suitable 
processing equipment to perform these operations. 

The data transmission network may consist of optical scanners, 
teletypes, or other I/O equipment for inputting data with the system from 
a school site and for transmitting computer-generated reports via phone 
lines to the school site, A detailed analysis of the different hardware 
configurations are discussed in a related paper,* 

Another major developmental requirement is that of generating the 
program that will execute the specific data processing tasks. The 
ccmputer program is the heart of the computer-based instructional 

*McManus, John F, Alternate Hardware Configurations for an Instructional 
Management System. Paper to be presented at 1971 AERA Convention. 
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monitoring and management system. To generate the necessary programs will 
require the strictest definition of the logical operations and analysis 
tasks to be performed. Creation of computer programs for an instructional 
management system, for example, will require explicit specification of the 
instructional decision rules and formating requirements to be used in the 
generation of prescriptions. 

Resource Allocation 

Computer compatibility does not necessarily imply computer desir- 
ability^ In education, money (or lack thereof) is a prime consideration 
in the implementation of any new program or system. Thus, the implemen- 
tation of a computer support system should be accomplished only after a 
detailed comparison has been made between the cost benefits of computer- 
supported and manual modes of operation. To accomplish this, questions 
such as the following, must be asked: Can the present staff perf >rm the 

desired functions adequately? If not, could they be performed by an 
expanded staff? If a computer-based system is to be used, will it be 
possible to purchase or lease computer hardware? Is the suftware 
available, and if not, is there enough time and money to adapt available 
programs or to develop new ones? What are the relative cost benefits 
of each approach? The answers to these questions will determine the 
appropriateness of a computer-managed Instructional system for the 
desired applications. 

In summary, prior to the development of computer-based instructional 
monitoring and management systems, the specific components requiring 
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computer support must be identified; the potential cost benefits of 
Such support must be analyzed, and appropriate hardware/soft'ware config- 
urations must be established. The most critical task, however, in the 
development of such systems, will be the development and validation of 
instructional decision rules. 




